feat: add config example validation script and workflow - #2627
feat: add config example validation script and workflow#2627patrick-stephens wants to merge 27 commits into
Conversation
📝 WalkthroughWalkthroughAdded configuration extraction and dry-run validation scripts, integrated them into pull-request workflow checks, and corrected configuration examples across installation and pipeline documentation. ChangesConfiguration validation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant PullRequest
participant GitHubActions
participant test-config.sh
participant FluentBitContainer
PullRequest->>GitHubActions: change Markdown files
GitHubActions->>test-config.sh: validate each changed file
test-config.sh->>FluentBitContainer: run configuration dry-run
FluentBitContainer-->>test-config.sh: return validation status
test-config.sh-->>GitHubActions: report failures or success
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
ba1f42c to
56630f1
Compare
eschabell
left a comment
There was a problem hiding this comment.
@patrick-stephens did some cleanup work but it now is good to go, thanks for this! Please merge this when you are ready?
|
Once merged we can see how things go and look to extend it in the future with checks for case, etc. |
There was a problem hiding this comment.
Actionable comments posted: 9
🧹 Nitpick comments (3)
scripts/test-config.sh (2)
119-128: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winAvoid rerunning the container just to capture failure output.
On failure, the container is run twice: once with output discarded (Line 120) to check the exit status, and again (Line 125) purely to surface output on stderr. Capture combined output on the first run instead, so a failing validation doesn't double the container startup/dry-run cost.
♻️ Proposed fix
- if ! $CONTAINER_RUNTIME run --rm -t -v "$OUTPUT_FILE":"$OUTPUT_FILE":ro "$VALIDATION_IMAGE" fluent-bit --dry-run --config="$OUTPUT_FILE" &>/dev/null; then + VALIDATION_OUTPUT=$($CONTAINER_RUNTIME run --rm -v "$OUTPUT_FILE":"$OUTPUT_FILE":ro "$VALIDATION_IMAGE" fluent-bit --dry-run --config="$OUTPUT_FILE" 2>&1) && VALIDATION_STATUS=0 || VALIDATION_STATUS=$? + if [ "$VALIDATION_STATUS" -ne 0 ]; then FAILED_VALIDATIONS+=("$LANGUAGE example $EXAMPLE_INDEX") - # Provide the configuration and failure output for debugging purposes on stderr echo "ERROR: Validation failed for $LANGUAGE example $EXAMPLE_INDEX in $FILE" >&2 cat "$OUTPUT_FILE" >&2 - $CONTAINER_RUNTIME run --rm -t -v "$OUTPUT_FILE":"$OUTPUT_FILE":ro "$VALIDATION_IMAGE" fluent-bit --dry-run --config="$OUTPUT_FILE" >&2 || true + echo "$VALIDATION_OUTPUT" >&2 else🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/test-config.sh` around lines 119 - 128, Update the validation command in the configuration-checking flow to capture its combined output during the initial container run while preserving the exit status. In the failure branch, print the captured output instead of invoking $CONTAINER_RUNTIME a second time; keep the existing configuration and failure-context diagnostics unchanged.
73-81: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winImage is pulled once per invocation, i.e., once per changed file per PR run.
The workflow invokes this script once per changed Markdown file (see
pr-example-validation.yaml, Line 58-61). Each invocation independently pulls$VALIDATION_IMAGE(Line 78), adding a network round-trip per file even when the image is already present locally. For PRs touching several documentation files, this adds up.Consider checking for a local image and only pulling when absent, or moving the pull into a one-time setup step.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/test-config.sh` around lines 73 - 81, Update the image setup around CONTAINER_RUNTIME and VALIDATION_IMAGE so the script checks whether VALIDATION_IMAGE is already available locally before pulling it. Invoke the runtime’s image-inspection command, pull only when the image is absent, and preserve the existing error message and exit behavior when a required pull fails.scripts/extract-config.sh (1)
146-171: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winSilent success when the tab title is never found.
In the
ENDblock, the error branch only fires whenfound_tab && !found_fence(Line 166). Ifwanted_titlenever matches anywhere in the file,found_tabstays0, so neither the success branch nor the error branch runs. The script exits0with empty output.
test-config.shmasks this because count mode already returns0and the caller skips extraction in that case. Butscripts/README.mddocuments this script for direct standalone use ("can also be called directly"). A typo'd tab title passed directly would silently succeed with no output instead of reporting an error.♻️ Proposed fix
} else if (found_tab && !found_fence) { printf "ERROR: %s code fence #%d not found in tab: %s\n", wanted_language, target_index, wanted_title > "/dev/stderr" exit 1 + } else if (!found_tab) { + printf "ERROR: tab not found: %s\n", wanted_title > "/dev/stderr" + exit 1 }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/extract-config.sh` around lines 146 - 171, Update the extract-mode validation in the END block so a missing tab title reports an error and exits nonzero instead of silently succeeding. Extend the existing found_tab/found_fence handling around wanted_title, while preserving count mode and the current missing-fence and missing-code-fence errors.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/pr-example-validation.yaml:
- Line 43: Update the changed_md grep filter in the workflow to use a literal
dot for the filename/path match, ensuring Markdown files at the repository root
and in subdirectories are detected while retaining the existing .md suffix
requirement.
- Line 43: Update the changed_md command in the PR validation workflow to use
git diff with the existing base and HEAD revision range, or first compute the
merge base before invoking git diff-tree. Preserve the current name and
diff-filter options so only added, modified, copied, or renamed Markdown files
are selected.
- Around line 42-46: Update the output-writing logic in the Markdown
change-detection step so the newline-separated value from changed_md uses GitHub
Actions’ multiline output delimiter syntax instead of echoing it as a plain
list= entry. Preserve the existing output name list and ensure the
delimiter-wrapped value is written safely to GITHUB_OUTPUT for multiple Markdown
files.
In `@pipeline/filters/parser.md`:
- Line 30: Update scripts/test-config.sh to discover each per-file tab title and
pass that title to extract-config.sh, rather than counting only
fluent-bit.yaml/fluent-bit.conf; ensure every renamed example is validated.
Apply this discovery behavior to the tab declarations at
pipeline/filters/parser.md:30-30 and :40-40, and
pipeline/outputs/kafka.md:110-110 and :157-157, using each declaration’s title
as the corresponding configuration name.
In `@pipeline/inputs/kafka.md`:
- Around line 145-156: Update the introductory paragraph to state that each
message is processed by the inline modify_kafka_message function rather than
kafka.lua, and revise the sentence structure so it clearly describes sending the
result back to the fb-sink topic on the same broker.
In `@pipeline/parsers.md`:
- Around line 68-72: Make both standalone YAML examples self-contained: in
pipeline/parsers.md, define custom_parser1 inline or reference a concrete
parser-file fixture before it is used; in pipeline/filters/kubernetes.md,
replace the legacy undefined parsers_file reference with the existing inline
custom-tag definition or another concrete parser-file fixture. Ensure each
extracted YAML tab validates independently.
In `@pipeline/router.md`:
- Line 210: Update the routing.yaml tab title in the routing documentation
around the routing.yaml example so scripts/test-config.sh recognizes and
extracts the routes configuration snippet. Use a supported extractor tab title
while preserving the existing routes syntax example.
In `@scripts/test-config.sh`:
- Around line 56-71: In the suppressed-file loop, replace the three redundant
conditions with one path-anchored match that accepts only an exact path or a
file beneath the suppressed entry as a directory. Preserve the informational
message and exit behavior, while ensuring names embedded in unrelated filenames
or extensions do not suppress validation.
- Around line 96-102: Update the Stage 1 counting flow around extract-config.sh
so genuine command failures remain visible and cause the validation to fail,
rather than being converted into EXAMPLE_COUNT=0. Preserve the legitimate
zero-example path that continues without validation, but capture and report the
extraction error separately using the existing script error-handling
conventions.
---
Nitpick comments:
In `@scripts/extract-config.sh`:
- Around line 146-171: Update the extract-mode validation in the END block so a
missing tab title reports an error and exits nonzero instead of silently
succeeding. Extend the existing found_tab/found_fence handling around
wanted_title, while preserving count mode and the current missing-fence and
missing-code-fence errors.
In `@scripts/test-config.sh`:
- Around line 119-128: Update the validation command in the
configuration-checking flow to capture its combined output during the initial
container run while preserving the exit status. In the failure branch, print the
captured output instead of invoking $CONTAINER_RUNTIME a second time; keep the
existing configuration and failure-context diagnostics unchanged.
- Around line 73-81: Update the image setup around CONTAINER_RUNTIME and
VALIDATION_IMAGE so the script checks whether VALIDATION_IMAGE is already
available locally before pulling it. Invoke the runtime’s image-inspection
command, pull only when the image is absent, and preserve the existing error
message and exit behavior when a required pull fails.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 526f19af-3dd1-4b14-80b6-0c240eb98fc1
📒 Files selected for processing (22)
.github/workflows/pr-example-validation.yamlinstallation/downloads/docker.mdpipeline/buffering.mdpipeline/filters/geoip2-filter.mdpipeline/filters/kubernetes.mdpipeline/filters/parser.mdpipeline/inputs/blob.mdpipeline/inputs/cpu-metrics.mdpipeline/inputs/kafka.mdpipeline/inputs/tail.mdpipeline/outputs/dynatrace.mdpipeline/outputs/gelf.mdpipeline/outputs/kafka.mdpipeline/outputs/loki.mdpipeline/outputs/s3.mdpipeline/parsers.mdpipeline/processors/conditional-processing.mdpipeline/processors/sql.mdpipeline/router.mdscripts/README.mdscripts/extract-config.shscripts/test-config.sh
💤 Files with no reviewable changes (2)
- pipeline/processors/conditional-processing.md
- pipeline/outputs/gelf.md
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
…in TOML format Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Signed-off-by: Patrick Stephens <pat@telemetryforge.io>
Resolve all Vale errors and suggestions in the config example validation PR, plus markdownlint errors in the new scripts README. - scripts/README.md: replace "e.g.," with "for example," (3 instances), convert 5 headings to sentence case, use "aren't" contraction, and fix markdownlint MD031/MD032/MD040 by adding blank lines around fences and lists and switching nested Markdown examples to 4-backtick outer fences - installation/downloads/docker.md: spell out "K8s" as "Kubernetes" - pipeline/inputs/tail.md: use "`inode` numbers" so the term is in code font and skipped by the spelling rule - pipeline/filters/parser.md: use "shouldn't" contraction - pipeline/outputs/kafka.md: use "isn't" contraction - pipeline/parsers.md: remove leading ellipsis and rewrite as a complete sentence Signed-off-by: Eric D. Schabell <eric@schabell.org>
The example validation job added in this branch never validated anything, and the script backing it could not run on macOS or report extraction failures. Five defects, none of which surface as a red build. Workflow (.github/workflows/pr-example-validation.yaml): - Replace git diff-tree with git diff for the changed-file list. diff-tree does not honor the three-dot merge-base range and returned an empty list, and without -r it is not recursive so it would have listed top-level directories rather than file paths. Drop --no-commit-id, which is a diff-tree-only option. --name-only and --diff-filter=AMCR are unchanged. - Match Markdown files on a literal .md suffix. The previous pattern './.*\.md$' left the dot unescaped, so it required every path to contain a directory separator and silently skipped Markdown files at the repository root. - Write the file list to GITHUB_OUTPUT using the multiline delimiter syntax with a random delimiter. As a plain list=value entry the runner parsed only the first line and rejected the rest with "Invalid format" as soon as a PR touched more than one file. Quote GITHUB_OUTPUT while here. - Update the comment that named git diff-tree. Script (scripts/test-config.sh): - Extract configurations into a per-run temporary directory and resolve its physical path before bind mounting. On macOS /tmp and $TMPDIR are symlinks into /private, which Docker Desktop file sharing cannot resolve, so it mounted an empty directory instead of the configuration and every example failed with a misleading "invalid YAML at line 1, column 1" error. The previous fixed filename also meant concurrent runs overwrote each other. Remove the directory on exit. - Check the exit status of the Stage 1 example count instead of discarding it with "2>/dev/null || echo 0". A genuine extraction failure became a count of zero, so the file was skipped as having no examples and the run reported success. Failures now go through the existing FAILED_VALIDATIONS list with the underlying stderr replayed, while a real zero count remains a legitimate skip. Signed-off-by: Eric D. Schabell <eric@schabell.org>
7e2c8c7 to
9fda3df
Compare
|
@patrick-stephens fixed a bunch of things that came up on the review of the review. See what you think? |
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
pipeline/outputs/s3.md (4)
154-182: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winValidate Parquet examples with an Arrow-enabled image or suppress them.
scripts/test-config.shusesfluent/fluent-bit:latestby default, and the official Fluent Bit image does not include Apache Arrow/Parquet support. The discoverablefluent-bit.yamlandfluent-bit.confexamples withformat: parquetcan fail duringfluent-bit --dry-run, so run them against an image with Parquet support, such asamazon/aws-for-fluent-bit, or addpipeline/outputs/s3.mdto the validator suppression list with an explicit reason.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pipeline/outputs/s3.md` around lines 154 - 182, Update the Parquet examples in the S3 output documentation so validation does not run them against the default image lacking Arrow/Parquet support. Either configure their validation to use an Arrow-enabled image such as amazon/aws-for-fluent-bit, or add pipeline/outputs/s3.md to the validator suppression list with an explicit reason, while preserving the examples themselves.Source: Learnings
171-182: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winParquet PutObject examples need explicit
total_file_size.These examples rely on the 100M default: the YAML example at 171-182, plus the migratory YAML and classic config examples around 844-882. Add
total_file_size: 50Mor another valid PutObject value so the examples document a supported explicit size.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pipeline/outputs/s3.md` around lines 171 - 182, Add an explicit valid total_file_size setting, such as 50M, to the Parquet PutObject examples in the shown YAML configuration and the related migratory YAML and classic configuration examples. Preserve the existing use_put_object settings and formatting while ensuring each relevant example documents the size explicitly.
49-54: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winClarify Arrow output and unset compression behavior.
compression: arrowis legacy compatibility syntax, not a normal codec. Useformat: arrowfor Arrow/Feather output, withcompression: zstdor unset for no Arrow compression. Addnone/unset to the S3compressiontable, and show Parquet files as uncompressed whenformat: parquetomitscompression.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pipeline/outputs/s3.md` around lines 49 - 54, The S3 documentation must clarify that Arrow output uses format: arrow, while compression: arrow is legacy syntax; update the compression table in pipeline/outputs/s3.md lines 49-54 to document none/unset, and revise pipeline/outputs/s3.md line 778 to show Parquet files are uncompressed when format: parquet omits compression, with Arrow compression using zstd or unset for no compression.
194-199: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winAlign the migration example with the documented PutObject requirement.
The Parquet section says
format parquetrequiresuse_put_object On, but lines 194-199 omit that setting. Adduse_put_object: onor update this block to state thatformat parquetenables it automatically.Proposed fix
**After (recommended):** ```yaml +use_put_object: on format: parquet compression: snappy🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pipeline/outputs/s3.md` around lines 194 - 199, Update the Parquet migration YAML example to include the documented use_put_object: on setting alongside format: parquet and compression: snappy, unless the surrounding documentation explicitly establishes that Parquet enables it automatically.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/pr-example-validation.yaml:
- Around line 65-68: Update the changed-file iteration around CHANGED_MD_FILES
to read one line at a time, preserving each newline-delimited path as a single
argument to ./scripts/test-config.sh. Replace the unquoted for loop while
retaining the existing processing message and error_count increment behavior.
- Line 43: Update the changed-file collection around changed_md so git diff runs
separately and its failure causes the validation step to exit nonzero; only
after a successful diff should its output be filtered for .md files, preserving
the existing no-Markdown-files behavior for an empty result.
In `@scripts/README.md`:
- Around line 41-49: Update the all-files validation example to track whether
any invocation of test-config.sh fails, while continuing to process every
Markdown file; after the loop completes, exit with the tracked failure status so
automation receives a non-zero result when any file fails.
---
Outside diff comments:
In `@pipeline/outputs/s3.md`:
- Around line 154-182: Update the Parquet examples in the S3 output
documentation so validation does not run them against the default image lacking
Arrow/Parquet support. Either configure their validation to use an Arrow-enabled
image such as amazon/aws-for-fluent-bit, or add pipeline/outputs/s3.md to the
validator suppression list with an explicit reason, while preserving the
examples themselves.
- Around line 171-182: Add an explicit valid total_file_size setting, such as
50M, to the Parquet PutObject examples in the shown YAML configuration and the
related migratory YAML and classic configuration examples. Preserve the existing
use_put_object settings and formatting while ensuring each relevant example
documents the size explicitly.
- Around line 49-54: The S3 documentation must clarify that Arrow output uses
format: arrow, while compression: arrow is legacy syntax; update the compression
table in pipeline/outputs/s3.md lines 49-54 to document none/unset, and revise
pipeline/outputs/s3.md line 778 to show Parquet files are uncompressed when
format: parquet omits compression, with Arrow compression using zstd or unset
for no compression.
- Around line 194-199: Update the Parquet migration YAML example to include the
documented use_put_object: on setting alongside format: parquet and compression:
snappy, unless the surrounding documentation explicitly establishes that Parquet
enables it automatically.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 0818df98-a13a-4d95-b0be-3d1bad914208
📒 Files selected for processing (22)
.github/workflows/pr-example-validation.yamlinstallation/downloads/docker.mdpipeline/buffering.mdpipeline/filters/geoip2-filter.mdpipeline/filters/kubernetes.mdpipeline/filters/parser.mdpipeline/inputs/blob.mdpipeline/inputs/cpu-metrics.mdpipeline/inputs/kafka.mdpipeline/inputs/tail.mdpipeline/outputs/dynatrace.mdpipeline/outputs/gelf.mdpipeline/outputs/kafka.mdpipeline/outputs/loki.mdpipeline/outputs/s3.mdpipeline/parsers.mdpipeline/processors/conditional-processing.mdpipeline/processors/sql.mdpipeline/router.mdscripts/README.mdscripts/extract-config.shscripts/test-config.sh
💤 Files with no reviewable changes (2)
- pipeline/outputs/gelf.md
- pipeline/processors/conditional-processing.md
🚧 Files skipped from review as they are similar to previous changes (17)
- pipeline/filters/kubernetes.md
- installation/downloads/docker.md
- pipeline/inputs/blob.md
- pipeline/processors/sql.md
- pipeline/filters/geoip2-filter.md
- pipeline/buffering.md
- scripts/extract-config.sh
- pipeline/outputs/loki.md
- pipeline/inputs/cpu-metrics.md
- pipeline/parsers.md
- pipeline/filters/parser.md
- pipeline/inputs/tail.md
- pipeline/outputs/dynatrace.md
- pipeline/inputs/kafka.md
- pipeline/router.md
- scripts/test-config.sh
- pipeline/outputs/kafka.md
| # We grep for *.md files you are interested in. | ||
| # The 'docs/' prefix is a common convention but adjust as necessary. | ||
| run: | | ||
| changed_md=$(git diff --name-only --diff-filter=AMCR ${{ github.event.pull_request.base.sha }}...HEAD | grep '\.md$' || true) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Workflow snippets:"
if [ -f .github/workflows/pr-example-validation.yaml ]; then
nl -ba .github/workflows/pr-example-validation.yaml | sed -n '1,140p'
else
echo "workflow file missing"
fi
echo
echo "Changed diff command occurrences:"
rg -n "changed_md=\\$\\(git diff|\\|\\| true|changed_files" .github/workflows/pr-example-validation.yaml || true
echo
echo "Behavioral sim for grep failure modes:"
bash -c 'set -o pipefail; failed=0; changed_md=$(echo "README.md" | grep -x "not-there" || true); echo "grep_missing_changed_md=(empty)=[$changed_md]"'
bash -c 'set -o pipefail; changed_md=$(rm -f /tmp/nowhere-$$ && cat /tmp/nowhere-$$ 2>/dev/null | grep -x "README.md" || true); echo "grep_diff_failed_changed_md=(empty)=[$changed_md]"; exit_code=$?; echo "pipeline_exit=$exit_code" '
bash -c 'git diff --name-only --diff-filter=AMCR invalid-sha...HEAD; echo "diff_invalid_exit=$?"; true'Repository: fluent/fluent-bit-docs
Length of output: 219
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Workflow snippets:"
if [ -f .github/workflows/pr-example-validation.yaml ]; then
awk '{printf "%6d\t%s\n", NR, $0}' .github/workflows/pr-example-validation.yaml | sed -n '1,140p'
else
echo "workflow file missing"
fi
echo
echo "Changed diff command occurrences:"
grep -n "changed_md=\$(git diff\|\\|\\| true\|changed_files" .github/workflows/pr-example-validation.yaml || true
echo
echo "Behavioral sim for grep failure modes:"
bash -c 'set -o pipefail; failed=0; changed_md=$(echo "README.md" | grep -x "not-there" || true); echo "grep_missing_changed_md=(empty)=[$changed_md]"'
bash -c 'set -o pipefail; changed_md=$(rm -f /tmp/nowhere-$$ && cat /tmp/nowhere-$$ 2>/dev/null | grep -x "README.md" || true); echo "grep_diff_failed_changed_md=$(cat)"
exit_code=${PIPESTATUS[@]:?}; echo "exit_code=$exit_code" '
bash -c 'set -o pipefail || true; git diff --name-only --diff-filter=AMCR invalid-sha...HEAD 2>/dev/null || echo "diff_invalid_exit_not_run"; true'Repository: fluent/fluent-bit-docs
Length of output: 7182
🏁 Script executed:
#!/bin/bash
set -u
echo "Git availability:"
command -v git || true
echo
echo "Diff with invalid SHA and git status:"
(git diff --name-only --diff-filter=AMCR invalid-sha...HEAD 2>&1 || true)
echo "status=$?"
echo
echo "Diff with valid refs when git is available:"
if command -v git >/dev/null 2>&1; then
git status --short --branch --porcelain
changed_md=$(git diff --name-only --diff-filter=AMCR HEAD...HEAD 2>&1 || true)
echo "exit=${PIPESTATUS[0]} changed_md=[$changed_md]"
fi
echo
echo "Bash pipefail/grep behavior:"
bash -c 'set -o pipefail; changed_md=$(printf "README.md\n" | grep -x "not-there" || true); echo "grep_missing_changed_md=[$changed_md]"'
bash -c 'set -e; changed_md=$(rm -f /tmp/nowhere-$$ && cat /tmp/nowhere-$$ 2>/dev/null | grep -x "README.md" || true); echo "should_not_print"; echo "changed_md=[$changed_md]"; false || true'
bash -c 'changed_md=$(rm -f /tmp/nowhere-$$ && cat /tmp/nowhere-$$ 2>/dev/null | grep -x "README.md" || true); echo "changed_md=[$changed_md]"; set -e; false'Repository: fluent/fluent-bit-docs
Length of output: 626
Fail when git diff cannot compute the changed-file list.
changed_md=$(git diff ... | grep '\.md$' || true) makes the whole command succeed even if git diff encounters an invalid revision or repository error. CHANGED_MD_FILES stays empty, so the validation step prints “No Markdown files changed” and exits successfully.
Compute git diff separately, exit on failure, then filter the output for Markdown files.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/pr-example-validation.yaml at line 43, Update the
changed-file collection around changed_md so git diff runs separately and its
failure causes the validation step to exit nonzero; only after a successful diff
should its output be filtered for .md files, preserving the existing
no-Markdown-files behavior for an empty result.
There was a problem hiding this comment.
I'm going to move this into the script itself so I can test it locally too I think rather than have to debug Github Actions...
There was a problem hiding this comment.
@patrick-stephens, that approach is suitable. Keep the workflow limited to passing the PR base SHA and target SHA to the script.
In the script, run git diff before the Markdown filter. If git diff fails, return a nonzero status. Filter for .md files only after the diff succeeds. Preserve the empty-result behavior when no Markdown files changed.
You are interacting with an AI system.
Cheers, yeah I've not run it on macOS and it's "special" shell so thanks for that. I'll re-run on Ubuntu to confirm all ok for CI and look to resolve that workflow issue which is outstanding. It may be we'll need to iterate a little after landing as testing workflows is unnecessarily difficult but using local scripts hopefully mitigates most of it. |
Resolves #2459 by providing a simple AWK based approach to validating configuration examples:
--dry-runthe configuration and report success/failureA local script is provided that can be run for any file in the repo or all of them:
A workflow is provided to run this for any files changed in a PR so if an update is made to documentation it should check it is valid.
There is a basic suppression approach by file as there are some valid reasons for this:
execare not part of the container imageAs part of these changes we also found failures in existing files that were resolved.
Some tweaks were also required, e.g. parser definition must be in a separate file for legacy TOML config so it was updated to be a comment for the examples (which would be rejected anyway otherwise).
There are options to use something more complex like markdown-tree or similar to build an AST from the Markdown file to then pull out the bits we need but this may not work with the specific Gitbook format anyway and requires a whole load of extra dependencies.
Currently there is an upstream failure with certain plugins triggering a segmentation fault for
--dry-run: fluent/fluent-bit#12113This is resolved by plugins: fix dry-run segmentation faults fluent-bit#12114 so waiting on that to merge.
Summary by CodeRabbit
Documentation
Validation